A Genetic Search In Policy Space For Solving Markov Decision Processes
نویسنده
چکیده
Markov Decision Processes (MDPs) have been studied extensively in the context of decision making under uncertainty. This paper presents a new methodology for solving MDPs, based on genetic algorithms. In particular, the importance of discounting in the new framework is dealt with and applied to a model problem. Comparison with the policy iteration algorithm from dynamic programming reveals the advantages and disadvantages of the proposed method.
منابع مشابه
Integrating value functions and policy search for continuous Markov Decision Processes
Value function approaches for Markov decision processes have been used successfully to find optimal policies for a large number of problems. Recent findings have demonstrated that policy search can be used effectively in reinforcement learning when standard value function techniques become overwhelmed by the size and dimensionality of the state space. We demonstrate that substantial benefits ca...
متن کاملAn Evolutionary Random Policy Search Algorithm for Solving Markov Decision Processes
T paper presents a new randomized search method called evolutionary random policy search (ERPS) for solving infinite-horizon discounted-cost Markov-decision-process (MDP) problems. The algorithm is particularly targeted at problems with large or uncountable action spaces. ERPS approaches a given MDP by iteratively dividing it into a sequence of smaller, random, sub-MDP problems based on informa...
متن کاملAccelerated decomposition techniques for large discounted Markov decision processes
Many hierarchical techniques to solve large Markov decision processes (MDPs) are based on the partition of the state space into strongly connected components (SCCs) that can be classified into some levels. In each level, smaller problems named restricted MDPs are solved, and then these partial solutions are combined to obtain the global solution. In this paper, we first propose a novel algorith...
متن کاملGenetic Programming as Policy Search in Markov Decision Processes
In this paper, we examine genetic programming as a policy search technique for planning problems representable as Markov Decision Processes. The planning task under consideration is derived from a real-time strategy war game. This problem presents unique challenges for standard genetic programming approaches; despite this, we show that genetic programming produces results competitive with stand...
متن کاملSolving POMDPs by Searching in Policy Space
Most algorithms for solving POMDPs itera tively improve a value function that implic itly represents a policy and are said to search in value function space. This paper presents an approach to solving POMDPs that repre sents a policy explicitly as a finite-state con troller and iteratively improves the controller by search in policy space. Two related al gorithms illustrate this approach. ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2002